<!DOCTYPE html>
<!--Author: Pranav Rajpurkar 2016-->
<html>
 <head>
  <meta charset="utf-8" />
  <title>KGE Editing</title>
  <meta name="description" content="KGE Editing Task." />
  <meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1" />
  <meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no" />
  <meta property="og:image" content="./deltakg_logo.png" />
  <link rel="image_src" type="image/png" href="./deltakg_logo.png" />
  <link rel="shortcut icon" href="./deltakg_logo.ico" type="image/x-icon" />
  <link rel="icon" href="./deltakg_logo.ico" type="image/x-icon" />
  <link rel="stylesheet" href="./bower_components/bootstrap/dist/css/bootstrap.min.css" />
  <link rel="stylesheet" href="./stylesheets/layout.css" />
  <link rel="stylesheet" href="./stylesheets/index.css" />
  <script async="" defer="" src="https://buttons.github.io/buttons.js"></script>
  <script src="./javascripts/analytics.js"></script>
 </head>
 <body>
  <div class="navbar navbar-default navbar-fixed-top" id="topNavbar" role="navigation">
   <div class="container clearfix" id="navContainer">
    <div class="rightNav">
     <div class="collapseDiv">
      <button class="navbar-toggle collapsed" type="button" data-toggle="collapse" data-target="#navbar" aria-expanded="false" aria-controls="navbar"><span class="glyphicon glyphicon-menu-hamburger"></span></button>
     </div>
     <div class="collapse navbar-collapse" id="navbar">
      <ul class="nav navbar-nav navbar-right">
       <li><a href="./index.html">Home</a></li>
       <li><a href="https://github.com/zjunlp/PromptKG/tree/main/deltaKG">Github</a></li>
       <li><a href="https://huggingface.co/spaces/zjunlp/KGEditor">Demo</a></li>
      </ul>
     </div>
    </div>
    <div class="leftNav">
     <div class="brandDiv">
      <a class="navbar-brand" href="./index.html">KGE Editing</a>
     </div>
    </div>
   </div>
  </div>
  <div class="cover" id="topCover">
   <div class="container">
    <div class="row">
     <div class="col-md-12">
      <h1 id="appTitle">KGE Editing</h1>
      <h2 id="appSubtitle">Language Model-based Knowledge Graph Embedding Editing</h2>
     </div>
    </div>
   </div>
  </div>
  <div class="cover" id="contentCover">
   <div class="container">
    <div class="row">
     <div class="col-md-5">
      <div class="infoCard">
       <div class="infoBody">
        <div class="infoHeadline">
         <h2>What is KGE Editing?</h2>
        </div>
        <p>The purpose of the KGE Editing task is to modify the erroneous knowledge in the KGE model and to inject 
            new knowledge into the KGE model. Thus, in response to the task objectives, we design two subtasks 
            (<i>EDIT</i> & <i>ADD</i>). For the <i>EDIT</i> sub-task, we edit the wrong fact knowledge that is 
            stored in the KG embeddings. Also, for the <i>ADD</i> sub-task, we add brand-new knowledge into the 
            model without re-training the whole model. 
        </p>
        <hr />
        <p><b>KGEditor</b> <span><b>K</b>nowledge <b>G</b>raph <b>E</b>mbeddings <b>E</b>ditor (KGEditor) </span>
            is a strong baseline for editing the knowledge in KGE model. KGEditor utilizes additional parameters 
            via the hyper networks. We construct an additional layer with the same architecture of FFN and leverage 
            its parameters for knowledge editing.
        </p>
        <a class="btn actionBtn" href="./index.html">More About KGEditor</a>
        <!-- <a class="btn actionBtn" href="http://arxiv.org/abs/1806.03822">SQuAD2.0 paper (Rajpurkar &amp; Jia et al. '18)</a> -->
        <hr />
        <p><b>Dataset </b>We build four datasets for the sub-task of EDIT and ADD based on two benchmark datasets 
            FB15k-237, and WN18RR. Firstly, we train KG embedding models with language models. For EDIT sub-task, we sample 
            some hard triples as candidates. For the ADD sub-task, we leverage the original 
            training set of FB15k-237 and WN18RR to build the pre-train dataset (original pre-train data) and use the data 
            from the standard inductive setting as they are not seen before. For more tails about the task and the datasets, please refer to our paper:
        </p>
        <!-- <a class="btn actionBtn" href="/LeaderBoard/explore/1.1/dev/">Explore SQuAD1.1 and model predictions</a> -->
        <a class="btn actionBtn" href="https://arxiv.org/abs/2301.10405">KGEditor paper (Siyuan Cheng et al. '22)</a>
        <div class="infoHeadline">
         <h2>Getting Started</h2>
        </div>
        <p>We release all the datasets and the pre-trained KGE model for the community.</p>
        <!-- <p> Download a copy of the dataset (distributed under the <a href="http://creativecommons.org/licenses/by-sa/4.0/legalcode">CC BY-SA 4.0</a> license):</p> -->
        <ul class="list-unstyled">
         <li><a class="btn actionBtn inverseBtn" href="https://huggingface.co/datasets/zjunlp/KGEditor" download="">Dataset (60 MB)</a></li>
         <li><a class="btn actionBtn inverseBtn" href="https://huggingface.co/zjunlp/KGEditor" download="">Pre-trained KGE Model (2.1 GB)</a></li>
        </ul>
        <p></p>
        <p>Take E-FB15k237 as an example, to evaluate your models, you should load the pre-trained KGE model contain the erroneous knowledge. 
            While testing the edit test dataset, you need to verify that the stable dataset after the model data parameters are changed.
        </p>
        <div class="infoHeadline">
         <h2>Have Questions?</h2>
        </div>
        <p>
           If you have any question about the KGE Editing task and the datasets, or you want to submit your results to update the leaderboard, feel free
            to emtail to us:
        </p> 
        <br>
        <ul>
            <li style="color: rgb(226, 163, 152);">
                <a href="mailto:zhangningyu@zju.edu.cn" tyle="font-size: 0.95rem; color:darkred;">
                    zhangningyu@zju.edu.cn;
                </a>
            </li>
            <li style="color: rgb(226, 163, 152);">
                <a href="mailto:sycheng@zju.edu.cn" tyle="font-size: 0.95rem; color:darkred;">
                    sycheng@zju.edu.cn;
                </a>
            </li>
        </ul>
        <br>
        <p>If you use or extend our work, please cite the paper as follows:</p>
        <br>
        <pre style="font-size: 1.0rem; background-color: rgb(245, 238, 238);">@article{DBLP:journals/corr/abs-2301-10405,
    author    = {Siyuan Cheng and Ningyu Zhang and Bozhong Tian and Zelin Dai and Feiyu Xiong and Wei Guo and Huajun Chen},
    title     = {Editing Language Model-based Knowledge Graph Embeddings},
    journal   = {CoRR},
    volume    = {abs/2301.10405},
    year      = {2023},
    eprinttype = {arXiv},
    eprint    = {2301.10405},
}</pre>
       </div>
       <div class="infoSubheadline">
       </div>
      </div>
     </div>
     <div class="col-md-7">
        <div class="card-header" style="padding-bottom: 0;">
            <button id="show-table1" class="show-table-btn active">E-FB15k237</button>
            <button id="show-table2" class="show-table-btn">E-WN18RR</button>
            <button id="show-table3" class="show-table-btn">A-FB15k237</button>
            <button id="show-table4" class="show-table-btn">A-WN18RR</button>
        </div>
        <div class="infoCard">
        <div class="infoBody">
            <table class="table performanceTable" id="table-1" style="display: block;">
           <tbody>
            <tr>
                <th>Rank</th>
                <th>Model</th>
                <th>Succ@1</th>
                <th>Succ@3</th>
                <th>ER<sub>roc</sub></th>
                <th>RK@3</th>
                <th>RK<sub>roc</sub></th>
            </tr>
            <tr>
                <td> <p></p><span class="date label label-default"></span></td>
                <td style="word-break:break-word;">Finetune<p class="institution"></p></td>
                <td><b>0.472</b></td>
                <td><b>0.746</b></td>
                <td><b>0.998</b></td>
                <td><b>0.543</b></td>
                <td><b>0.977</b></td>
            </tr>
            
            <tr>
                <td> <p>1</p><span class="date label label-default">Jan 19, 2022</span></td>
                <td style="word-break:break-word;">KGEditor<p class="institution"><a class="link" href="https://arxiv.org/abs/2301.10405" style="font-style: italic;">Arxiv'23</p></a></td>
                <td><b>0.866</b></td>
                <td><b>0.986</b></td>
                <td><b>0.999</b></td>
                <td><b>0.874</b></td>
                <td><b>0.635</b></td>
            </tr>
            <tr>
                <td> <p>2</p><span class="date label label-default">Jan 19, 2022</span></td>
                <td style="word-break:break-word;">Knowledge Editor<p class="institution"><a class="link" href="https://doi.org/10.18653/v1/2021.emnlp-main.522" style="font-style: italic;">EMNLP'21</a></p></td>
                <td><b>0.702</b></td>
                <td><b>0.969</b></td>
                <td><b>0.999</b></td>
                <td><b>0.912</b></td>
                <td><b>0.685</b></td>
            </tr>
            <tr>
                <td> <p>3</p><span class="date label label-default">Jan 19, 2022</span></td>
                <td style="word-break:break-word;">MEND<a class="link" href="https://openreview.net/forum?id=0DcZxeWfOPt"><p class="institution">ICLR'22</p></a></td>
                <td><b>0.828</b></td>
                <td><b>0.950</b></td>
                <td><b>0.954</b></td>
                <td><b>0.750</b></td>
                <td><b>0.993</b></td>
            </tr>
            <tr>
                <td> <p>4</p><span class="date label label-default">Jan 19, 2022</span></td>
                <td style="word-break:break-word;">CALINET<a class="link" href="https://doi.org/10.48550/arXiv.2210.03329"><p class="institution">EMNLP'22</p></a></td>
                <td><b>0.328</b></td>
                <td><b>0.348</b></td>
                <td><b>0.937</b></td>
                <td><b>0.353</b></td>
                <td><b>0.997</b></td>
            </tr>
            <tr>
                <td> <p>5</p><span class="date label label-default">Jan 19, 2022</span></td>
                <td style="word-break:break-word;">K-Adapter<a class="link" href="https://arxiv.org/abs/2002.01808"><p class="institution">ACL'21</p></a></td>
                <td><b>0.329</b></td>
                <td><b>0.348</b></td>
                <td><b>0.926</b></td>
                <td><b>0.001</b></td>
                <td><b>0.999</b></td>
            </tr>
            <tr>
                <td> <p></p><span class="date label label-default"></span></td>
                <td style="word-break:break-word;">Zero-Shot Learning<p class="institution"></p></td>
                <td><b>0.000</b></td>
                <td><b>0.000</b></td>
                <td><b>-</b></td>
                <td><b>1.000</b></td>
                <td><b>0.000</b></td>
            </tr>
           </tbody>
            </table>
            <table class="table performanceTable" id="table-2" style="display: none;">
                <tbody>
                <tr>
                    <th>Rank</th>
                    <th>Model</th>
                    <th>Succ@1</th>
                    <th>Succ@3</th>
                    <th>ER<sub>roc</sub></th>
                    <th>RK@3</th>
                    <th>RK<sub>roc</sub></th>
                </tr>
                <tr>
                    <td> <p></p><span class="date label label-default"></span></td>
                    <td style="word-break:break-word;">Finetune<p class="institution"></p></td>
                    <td><b>0.758</b></td>
                    <td><b>0.863</b></td>
                    <td><b>0.998</b></td>
                    <td><b>0.847</b></td>
                    <td><b>0.746</b></td>
                </tr>
                <tr>
                    <td> <p>1</p><span class="date label label-default">Jan 19, 2022</span></td>
                    <td style="word-break:break-word;">KGEditor<p class="institution"><a class="link" href="https://arxiv.org/abs/2301.10405" style="font-style: italic;">Arxiv'23</p></a></td>
                    <td><b>0.833</b></td>
                    <td><b>0.844</b></td>
                    <td><b>0.991</b></td>
                    <td><b>0.956</b></td>
                    <td><b>0.256</b></td>
                </tr>
                <tr>
                    <td> <p>2</p><span class="date label label-default">Jan 19, 2022</span></td>
                    <td style="word-break:break-word;">MEND<a class="link" href="https://openreview.net/forum?id=0DcZxeWfOPt"><p class="institution">ICLR'22</p></a></td>
                    <td><b>0.815</b></td>
                    <td><b>0.827</b></td>
                    <td><b>0.948</b></td>
                    <td><b>0.957</b></td>
                    <td><b>0.772</b></td>
                </tr>
                <tr>
                    <td> <p>3</p><span class="date label label-default">Jan 19, 2022</span></td>
                    <td style="word-break:break-word;">Knowledge Editor<p class="institution"><a class="link" href="https://arxiv.org/abs/2301.10405" style="font-style: italic;">EMNLP'21</p></a></td>
                    <td><b>0.599</b></td>
                    <td><b>0.682</b></td>
                    <td><b>0.978</b></td>
                    <td><b>0.935</b></td>
                    <td><b>0.041</b></td>
                </tr>
                <tr>
                    <td> <p>4</p><span class="date label label-default">Jan 19, 2022</span></td>
                    <td style="word-break:break-word;">K-Adapter<a class="link" href="https://arxiv.org/abs/2002.01808"><p class="institution">ACL'21</p></a></td>
                    <td><b>0.638</b></td>
                    <td><b>0.752</b></td>
                    <td><b>0.992</b></td>
                    <td><b>0.009</b></td>
                    <td><b>0.999</b></td>
                </tr>
                <tr>
                    <td> <p>5</p><span class="date label label-default">Jan 19, 2022</span></td>
                    <td style="word-break:break-word;">CALINET<a class="link" href="https://doi.org/10.48550/arXiv.2210.03329"><p class="institution">EMNLP'22</p></a></td>
                    <td><b>0.538</b></td>
                    <td><b>0.649</b></td>
                    <td><b>0.991</b></td>
                    <td><b>0.446</b></td>
                    <td><b>0.994</b></td>
                </tr>
                <tr>
                    <td> <p></p><span class="date label label-default"></span></td>
                    <td style="word-break:break-word;">Zero-Shot Learning<p class="institution"></p></td>
                    <td><b>0.000</b></td>
                    <td><b>0.000</b></td>
                    <td><b>-</b></td>
                    <td><b>1.000</b></td>
                    <td><b>0.000</b></td>
                </tr>
                </tbody>
            </table>
            <table class="table performanceTable" id="table-3" style="display: none;">
            <tbody>
             <tr>
                 <th>Rank</th>
                 <th>Model</th>
                 <th>Succ@1</th>
                 <th>Succ@3</th>
                 <th>ER<sub>roc</sub></th>
                 <th>RK@3</th>
                 <th>RK<sub>roc</sub></th>
             </tr>
             <tr>
                 <td> <p></p><span class="date label label-default"></span></td>
                 <td style="word-break:break-word;">Finetune<p class="institution"></p></td>
                 <td><b>0.906</b></td>
                 <td><b>0.976</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.223</b></td>
                 <td><b>0.977</b></td>
             </tr>
             <tr>
                 <td> <p>1</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">KGEditor<p class="institution"><a class="link" href="https://arxiv.org/abs/2301.10405" style="font-style: italic;">Arxiv'23</p></a></td>
                 <td><b>0.796</b></td>
                 <td><b>0.923</b></td>
                 <td><b>0.998</b></td>
                 <td><b>0.899</b></td>
                 <td><b>0.920</b></td>
             </tr>
             <tr>
                 <td> <p>2</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">Knowledge Editor<p class="institution"><a class="link" href="https://arxiv.org/abs/2301.10405" style="font-style: italic;">EMNLP'21</p></a></td>
                 <td><b>0.648</b></td>
                 <td><b>0.884</b></td>
                 <td><b>0.997</b></td>
                 <td><b>0.926</b></td>
                 <td><b>0.971</b></td>
             </tr>
             <tr>
                 <td> <p>3</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">MEND<a class="link" href="https://openreview.net/forum?id=0DcZxeWfOPt"><p class="institution">ICLR'22</p></a></td>
                 <td><b>0.517</b></td>
                 <td><b>0.745</b></td>
                 <td><b>0.991</b></td>
                 <td><b>0.499</b></td>
                 <td><b>0.977</b></td>
             </tr>
             <tr>
                 <td> <p>4</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">K-Adapter<a class="link" href="https://arxiv.org/abs/2002.01808"><p class="institution">ACL'21</p></a></td>
                 <td><b>0.871</b></td>
                 <td><b>0.981</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.000</b></td>
                 <td><b>0.999</b></td>
             </tr>
             <tr>
                 <td> <p>5</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">CALINET<a class="link" href="https://doi.org/10.48550/arXiv.2210.03329"><p class="institution">EMNLP'22</p></a></td>
                 <td><b>0.714</b></td>
                 <td><b>0.870</b></td>
                 <td><b>0.997</b></td>
                 <td><b>0.034</b></td>
                 <td><b>0.999</b></td>
             </tr>
             <tr>
                <td> <p></p><span class="date label label-default"></span></td>
                <td style="word-break:break-word;">Zero-Shot Learning<p class="institution"></p></td>
                <td><b>0.000</b></td>
                <td><b>0.000</b></td>
                <td><b>-</b></td>
                <td><b>1.000</b></td>
                <td><b>0.000</b></td>
            </tr>
            </tbody>
            </table>
            <table class="table performanceTable" id="table-4" style="display: none;">
            <tbody>
             <tr>
                 <th>Rank</th>
                 <th>Model</th>
                 <th>Succ@1</th>
                 <th>Succ@3</th>
                 <th>ER<sub>roc</sub></th>
                 <th>RK@3</th>
                 <th>RK<sub>roc</sub></th>
             </tr>
             <tr>
                 <td> <p></p><span class="date label label-default"></span></td>
                 <td style="word-break:break-word;">Finetune<p class="institution"></p></td>
                 <td><b>0.997</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.554</b></td>
                 <td><b>0.996</b></td>
             </tr>
             <tr>
                 <td> <p>1</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">KGEditor<p class="institution"><a class="link" href="https://arxiv.org/abs/2301.10405" style="font-style: italic;">Arxiv'23</p></a></td>
                 <td><b>0.998</b></td>
                 <td><b>1.0</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.956</b></td>
                 <td><b>0.300</b></td>
             </tr>
             <tr>
                 <td> <p>2</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">MEND<a class="link" href="https://openreview.net/forum?id=0DcZxeWfOPt"><p class="institution">ICLR'22</p></a></td>
                 <td><b>0.999</b></td>
                 <td><b>1.0</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.810</b></td>
                 <td><b>0.987</b></td>
             </tr>
             <tr>
                 <td> <p>3</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">Knowledge Editor<p class="institution"><a class="link" href="https://arxiv.org/abs/2301.10405" style="font-style: italic;">EMNLP'21</p></a></td>
                 <td><b>0.986</b></td>
                 <td><b>0.996</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.975</b></td>
                 <td><b>0.090</b></td>
             </tr>
             <tr>
                 <td> <p>4</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">CALINET<a class="link" href="https://doi.org/10.48550/arXiv.2210.03329"><p class="institution">EMNLP'22</p></a></td>
                 <td><b>0.832</b></td>
                 <td><b>0.913</b></td>
                 <td><b>0.995</b></td>
                 <td><b>0.511</b></td>
                 <td><b>0.987</b></td>
             </tr>
             <tr>
                 <td> <p>5</p><span class="date label label-default">Jan 19, 2022</span></td>
                 <td style="word-break:break-word;">K-Adapter<a class="link" href="https://arxiv.org/abs/2002.01808"><p class="institution">ACL'21</p></a></td>
                 <td><b>0.898</b></td>
                 <td><b>0.978</b></td>
                 <td><b>0.999</b></td>
                 <td><b>0.002</b></td>
                 <td><b>0.999</b></td>
             </tr>
             <tr>
                <td> <p></p><span class="date label label-default"></span></td>
                <td style="word-break:break-word;">Zero-Shot Learning<p class="institution"></p></td>
                <td><b>0.000</b></td>
                <td><b>0.000</b></td>
                <td><b>-</b></td>
                <td><b>1.000</b></td>
                <td><b>0.000</b></td>
            </tr>
            </tbody>
            </table>
        </div>
        </div>
       </div>
    </div>
   </div>
  </div>
  <nav class="navbar navbar-default navbar-static-bottom footer">
   <div class="container clearfix">
    <div class="rightNav">
     <div>
      <ul class="nav navbar-nav navbar-right">
       <li><a href="./index.html">KGEditor</a></li>
       <li><a href="https://github.com/zjunlp">ZJUNLP Group</a></li>
      </ul>
     </div>
    </div>
   </div>
  </nav>
  <script src="./bower_components/jquery/dist/jquery.min.js"></script>
  <script src="./bower_components/bootstrap/dist/js/bootstrap.min.js"></script>
  <script type="text/javascript">
    var buttonArr = document.getElementsByClassName("show-table-btn");
    var TableArr = document.getElementsByClassName("table performanceTable");
    console.log(buttonArr);
    for(var i = 0; i < buttonArr.length; i++) {
        buttonArr[i].onclick = function() {
            for(var j = 0; j < buttonArr.length; j++) {
                TableArr[j].style.display = "none";
                buttonArr[j].className = "show-table-btn";
                if(this == buttonArr[j]) {
                    buttonArr[j].className = "show-table-btn active";
                    TableArr[j].style.display = "block";
                }
            }
        }
    }
    </script>
 </body>
</html>
